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Abstract — In this note, we show that the opera- 
tor theoretic concept of Kolmogorov numbers and 
the number of degrees of freedom at level e of a 
communication channel are closely related. Linear 
communication channels may be modeled using lin- 
ear compact operators on Banach or Hilbert spaces 
and the number of degrees of freedom of such 
channels is defined to be the number of linearly 
independent signals that may be communicated over 
this channel, where the channel is restricted by a 
threshold noise level. Kolmogorov numbers are a 
particular example of s-numbers, which are defined 
over the class of bounded operators between Banach 
spaces. We demonstrate that these two concepts are 
closely related, namely that the Kolmogorov numbers 
correspond to the "jump points" in the function 
relating numbers of degrees of freedom with the noise 
level e. We also establish a useful numerical compu- 
tation result for evaluating Kolmogorov numbers of 
compact operators. 

I. Introduction 

The number of degrees of freedom of a communi- 
cation channel plays an important role in evaluating 
the channel's Shannon capacity (see e.g. [1]). In 
a physical communication system, because of con- 



straints such as finite transmission power and noise at 
the receiver, only finitely many (linearly independent) 
signals may be exchanged between the transmitter 
and receiver. This physical intuition may be captured 
using the concept of number of degrees of freedom of 
the communication channel. The number of degrees 
of freedom of a communication channel has been 
used in evaluating the Shannon capacity for several 
physically realistic channels 1 (see e.g. [1], [2], [3], 
[4]). This concept has also been used in various other 
problem domains such as multi-antenna communica- 
tion [5], [6], [7], optics [8], [9] and electromagnetic 
field sampling [10]. 

Kolmogorov numbers are a particular example of 
so called s-number sequences. In the theory of s- 
numbers one associates with every bounded linear 
operator T, mapping between any two Banach spaces, 
a scalar sequence si(T) > s 2 (T) > ... > (see 
e.g. [11], [12]). A classical example in the more 

'The concept of number of degrees of freedom is used 
implicitly in evaluating the capacity of Shannon's classical 
bandwidth limited, additive Gaussian white noise channel. In 
particular it can be shown that as the bandwidth becomes large 
the number of degrees of freedom at level e approaches the well 
know constant 2WT, for all noise levels e (see Gallager [4, 
Ch. 8]). 
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restricted category of compact operators mapping 
between Hilbert spaces is the sequence of singular 
values, s-number sequences of various types have 
primarily been used to classify operators based on 
the behaviour of the sequence s n as n -> oo. In par- 
ticular, various interesting operator ideals have been 
obtained based on the behaviour of these sequences. 

In this note we establish the connection between 
Kolmogorov numbers and degrees of freedom of a 
communication channel. The remainder of this note 
is organised as follows: in Section II, we recall the 
definitions of degrees of freedom of a communication 
channel and briefly explain the physical intuition 
behind the definition. We also recall the definitions 
of s-number sequences in general and Kolmogorov 
numbers in particular. We present our main results, 
establishing the connection between the number of 
degrees of freedom and Kolmogorov numbers, in 
Section III and provide conclusions in the final 
Section IV. 

II. Definitions 

A. Degrees of Freedom 

The number of degrees of freedom of a communi- 
cation channel has for example been defined in [1]. 
The physical model used in [1] is as follows: there 
is a normed vector space X of transmitter functions 
and a normed vector space Y of receiver functions 
and a channel operator T : X — > Y that is a compact 
linear operator. Physically, the elements of X can be 
thought of as signals that a transmitter generates, 
and T maps these signals onto elements of Y, which 
is the space of signals a receiver can measure. It is 
further assumed that there are two constraints on 
this channel: 1) the power available for transmission 



is finite. This constraint is modeled by assuming that 
the transmitter may only generate elements of X 
that have a norm less than or equal to 1. 2) the 
receiver is noisy and therefore small signals can not 
be measured. This constraint is modeled by assuming 
that only received signals with norm greater than 
some pre-specifled small constant e may be measured. 

It was shown in [1] that the following definition 
may be used for the number of degrees of freedom 
of a compact operator that models a communication 
channel. 

Definition 2.1 (Degrees of freedom at level e): 
Suppose X and Y are normed spaces with norms 
|| • || jsc and || • respectively, and T : X — > Y is 
a compact operator. Also, let B r! x(0) denote the 
closed ball of radius r centered at 6 in X. Then the 
number of degrees of freedom Af(e) of T at level e 
is the smallest 2 JV e such that there exists a 
set of vectors {ipi, . . . , tp N } c Y such that for all 



inf oi, 



< e. 



It was shown in [1] that the number of degrees 
of freedom is well defined. Physically, we interpret 
this definition as follows: if there is some constraint 
|| • || x < 1 on the space of transmitter functions 3 and 
if the receiver can only measure signals that satisfy 
| || y > e, then the number of degrees of freedom 
is the maximum number of linearly independent 

2 Zq denotes the set of non-negative integers. 

3 We normalised the transmitter power constraint by as- 
suming that available transmission power, P = 1. Different 
normalisations can e.g. be achieved by re-scaling the norm on 
X. 
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signals that the receiver can distinguish under these 
constraints, where the maximum is taken over all 
possible signal constellations. 

The following theorem is a simple consequence of 
the above definition. 

Theorem 2.1: [1, Th. 3.2] Suppose X and Y 
are normed spaces with norms || \\x and || \\y, 
respectively, and T : X — > Y is a compact operator. 
Let Af(e) denote the number of degrees of freedom 
of T at level e. Then 

1) Af(e) = for all e > ||T||. 

2) Unless T is identically zero, there exists an e > 
such that Af(e) > 1 for all < e < e Q . 

3) M(e) is a non-increasing, upper semicontinu- 
ous function of e. 

4) In any finite interval (ei,e2) C K, with < 
ei < 62, A/"(e) has only finitely many disconti- 
nuities, i.e. Af(e) only takes finitely many non- 
negative integer values in any finite e interval. 

B. s-numbers and Kolmogorov numbers 

The axiomatic characterisation of s-numbers for 
Banach space valued operators is due to Pietsch, 
cf. [12]. In the remainder of this section, we assume 
that X, X', Y and Y' are Banach spaces, B(X, Y) de- 
notes the set of bounded linear operators T : X ->F 
and || | is the standard operator norm on B(X, Y). 
Also, we denote by I the identity operator. 

Definition 2.2: Let s : T ^ (s„(T))£° =1 be a 
rule that assigns to every bounded operator T on 
some Banach space a scalar sequence such that the 
following conditions are satisfied: 

SN1 ||T|| = si(T) > s 2 (T) > ... > 0. 
SN2 s n (S + T) < s n (S) + ||T||, for S,T e 
B(X,Y) and n= 1,2,.... 



SN3 s n (BTA) < \\B\\s n (T)\\A\\, for A e 
B(X',X),T g B{X,Y),B e B(Y,Y') and 
n = 1,2, .. .. 

SN4 s n (I) = 1, for n= 1,2,.... 

SN5 s„(T) = if rank(T) < n. 
Then (s„(T))^ =1 is called an s-number sequence. 
It turns out that in the compact, Hilbert space case, 
i.e. if we restrict the above definition to the class of 
compact operators mapping between Hilbert spaces, 
the above properties SN1-SN5 uniquely characterise 
the singular values of the operator (ordered in 
descending order). 

For any linear subspace S of Y let denote the 
natural surjection from Y onto the quotient space 
Y/S. The numbers 

d n (T) ±mi{\\QlT\\ :dim(S) < n} 

for T e B(X, Y) define an s-number sequence [12], 
and are called Kolmogorov numbers. 

III. Main results 

Let X and Y be Banach spaces with norms || • ||x 
and | \\y, respectively, let T : X — > Y be compact 4 
and suppose B(X) is the unit ball in X. Let M(e) 
denote the number of degrees of freedom of T at level 
e and let d n = d n (T) denote the n th Kolmogorov 
number. 

For ease of notaion we set Af(e) = oo for e < 
for the remainder of this document. Now, consider 

4 We only consider compact T, because the number of 
degrees of freedom is only defined for compact operators. One 
can conclude from the arguments in [1] that for physically real- 
istic models of communication channels, the channel operator 
must be compact. However, it may be possible to generalise 
the theory presented in the current paper to bounded operators 
that are not necessarily compact. 
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the sequence {a, n = a n (T)},n = 1,2,... implicitely 
defined as follows 

sup e>crn A/"(e) = n — 1 and 

inf £<(T „ 7V(e) = N > n. 

Further, if n < N then for all m such that n < m < 
N, a m = a n . The sequence {a n } was defined in [1] 
and cr„ was called the n th DOF singular value of T 
in [1]. It should be noted that in the Hilbert space 
situation, a n is simply the n th singular value of T. 
Also, 7V(e) is discontinuous exactly at e = a n for 

n = 1,2, We now show that a n is in fact equal to 

d n for which we will need the following elementary 
lemma. 

Lemma 3.1: 



+ 1 



inf sup inf 

Tpi,...,ip n eY X £B(X) a\,...,a n & 



Tx — aiipi 



Proof: Let S be any subspace of Y such that 
dim(S') < n. By the definition of Qg, 

||Q^T||= sup inf : HTz-j/Hy. 

The lemma now follows from two simple facts: for 
dim(5) < n there exists a set of vectors {V>i, ■■ .,ip n } 
such that they span S, and the dimension of the span 
of n vectors is not greater than n. ■ 

Theorem 3.2: a n = d n . 
Proof: Because a\ = d\ = \\T\\, we only need to 
prove that a n+1 = d n+ \ for n = 1, 2, 

Now suppose ctat = ctat-i = . . . = cr„ + i < a n . We 
first prove that a n+ i > d n+ \. Let 5 > be some 
arbitrary number satisfying 5 < o n - a n+ i. Then for 
e = + (5, 

AT(e) = n. 

Therefore, there exists a set {ipi, . . . ,ip n } e Y such 
that 



sup inf 



< e = <t„+i + 5. 



Because, the constant 5 > can be made arbitrarily 
small, 



Tx - ^2 aiipi 



d n+ i = inf sup inf 

4>i,...,i> n eY xe B(X) ai,...,a„el 

< CT„+1. (1) 

Relabeling N = n + 1, we get 

To prove the converse inequality, let e < a N . Then, 
7V(e) > N. Therefore, for all sets {ip u . . . , ip N } C Y, 
there exists an x e B(X) such that, 



inf 

ai ,. . . ,a^v 

Therefore, 



AT 



d n +l > djy 



inf sup inf 



= Cn+1- (2) 

Here, we used the s-number property SN1 that d n is 
non-increasing in n. The result now follows from (1) 
and (2). ■ 
Remark 3.1: The number of degrees of freedom 
of a communication channel is defined for compact 
operators mapping between arbitrary normed spaces, 
while the concept of Kolmogorov numbers is defined 
only for operators mapping between Banach spaces. 
However, if X and Y are normed spaces and T : 
X -> Y is compact, then consider the completions X 
and Y of X and Y, respectively, so that X and Y 
are Banach spaces. Let T : X -> Y be the standard 
extension of T from X to X. Then cr n (T) = cr„(T). 
This fact follows directly from the compactness of T 
and the definition of A/"( ). 



We also restate below a useful result in [1] that 
aids in the numerical computation of Kolgomorov 
numbers for compact operators. 

Theorem 3.3: [1, Th. 3.8] Suppose X and Y are 
Banach spaces and T : X — > Y is a compact operator. 
Also suppose that X has a complete Schauder basis 
{0i,02,...} and let S m = span{0i, . . . , m }. Let 
T m = T\ Sm : S m -> Y, m E Z+. If a n , the n th DOF 
singular value of T, exists then for m large enough 
(T„ im , the n th DOF singular value of T m , will exist 
and 

lim a nm = a n . 

m— > co 

If a n ,m exists then it is a lower bound for a n . 
We can therefore use finite dimensional approxima- 
tions of T to numerically compute the Kolgomorov 
numbers. 

IV. Conclusion 

In this note we establish the connection between 
Kolgomorov numbers and degrees of freedom of a 
communication channel. Specifically, we show that 
the jump points (discontinuities) of the function 
relating the number of degrees of freedom to the 
noise level e are equal to the Kolgomorov numbers. 
This connection invites the question as to whether 
other s-number sequences such as Gelfand numbers 
or approximation numbers play an equally important 
role in communication systems. 
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